Focus adjustment method, focus adjustment apparatus, and control method thereof

ABSTRACT

An apparatus comprises a face position detection section for detecting at least a position at which is present a person&#39;s face inside a frame using an image signal obtained from an image sensor for photoelectrically converting an object image captured by a photographing optical system, a focus control section for controlling the photographing optical system by referencing in-focus positions of the object image obtained inside focus detection areas for detecting focus states of the object image positioned in the frame, and a control section for controlling the focus control section so as to reference at least one or the other of an in-focus position obtained in a first focus detection area in which is present a person&#39;s face and an in-focus position obtained in a second focus detection area in which is expected to be present a body of a person determining from the position of the face of the person.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technology for performingauto-focusing face detection information.

2. Description of the Related Art

Conventionally, when photographing an object image with an auto-focuscamera in which the focus detection area is fixed in the middle of theframe, it has been necessary to place the object image in the center,engage the focus lock, and then compose the photograph.

A method has therefore been developed for setting a plurality of focusdetection areas inside the frame with the object of making auto-focuspossible without changing the composition of the photograph, namely,without using the focus lock. This technology has two methods, one forthe user to select one desired focus detection area from a plurality ofdetection areas, and one for estimating the main object from focusdetection results from the plurality of focus detection areas anddetermining the focal length to use. Of these, the latter method hasgreater convenience, as the user can photograph without worrying aboutselecting a focus detection area. However, it has the problem that theautomatically selected focus detection area does not necessarily detectthe focus for the object which the user wishes to photograph.

One means of solving this is an image capturing apparatus constituted soas to detect a person's face in image data, set a focus detection areaso as to include the detected face, and adjust the focus accordingly, inorder to refine the precision of the estimation discussed above bylimiting to humans. (See Japanese Patent Application Laid-Open No.2003-107335.) Japanese Patent Application Laid-Open No. 2003-107335describes an image capturing apparatus constituted so as to detect anobject's face in image data, set a focus detection area so as to includethe detected face, and adjust the focus accordingly, in order to refinethe precision of the estimation discussed above by limiting to the facesof objects. In Japanese Patent Application Laid-Open No. 2003-107335,the focusing action is performed using a detected face, so thepossibility of mistakenly focusing on other objects decreases.

However, with Japanese Patent Application Laid-Open No. 2003-107335,focus can sometimes be unadjusted in cases where the contrast of theface is low and sufficient focus detection data cannot be obtained, eventhough the face can be detected. Also, if objects such as flower vasesat differing distances in the focus detection area are closer to theimage capturing apparatus than a face, focus cannot be adjusted, eventhough a face can be detected.

SUMMARY OF THE INVENTION

Accordingly, the present invention was conceived with theabove-mentioned problems in mind, and makes it possible to focus on theobject intended by the user in a groundbreaking manner not possible withthe conventional technology.

In order to solve these problems and attain the object, according to afirst aspect of the present invention, a focus adjustment apparatuscomprises a face position detection unit adapted to detect a position atwhich exists at least a face of a person in the frame using imagesignals obtained from image sensors which photoelectrically convert anobject image captured by a photographing optical system, a focus controlunit adapted to control the photographing optical system by referencingan in-focus position of the object image obtained in a focus detectionarea for detecting the focus state of the object image positioned in theframe, a control unit adapted to control the focus control unit toreference at least either an in-focus position obtained in a first focusdetection area in which a person's face is present or an in-focusposition obtained in a second focus detection area in which it isestimated that a person's body is present determining from the positionof the person's face, and a determination unit adapted to determine thereliability with which the person's face can be focused on in the firstfocus detection area, the control unit controlling the focus controlunit to reference either the in-focus position obtained in the firstfocus detection area or the in-focus position obtained in the secondfocus detection area based the determination results of thedetermination unit.

According to a second aspect of the present invention, a focusadjustment apparatus comprises a face position detection unit adapted todetect a position at which exists at least a face of a person in theframe using image signals obtained from image sensors whichphotoelectrically convert an object image captured by a photographingoptical system, a focus control unit adapted to control thephotographing optical system by referencing an in-focus position of theobject image obtained in a focus detection area for detecting the focusstate of the object image positioned in the frame, and a control unitadapted to control the focus control unit to reference at least eitheran in-focus position obtained in a first focus detection area in which aperson's face is present or an in-focus position obtained in a secondfocus detection area in which it is estimated that a person's body ispresent determining from the position of the person's face, the controlunit controlling the focus control unit to reference the in-focusposition obtained in the first focus detection area if the differencebetween the in-focus position in the first focus detection area and thein-focus position in the second focus detection area is equal to or lessthan a pre-determined value.

According to a third aspect of the present invention, a focus adjustmentapparatus comprises a face position detection unit adapted to detect aposition at which exists at least a face of a person in the frame usingimage signals obtained from image sensors which photoelectricallyconvert an object image captured by a photographing optical system, afocus control unit adapted to control the photographing optical systemby referencing an in-focus position of the object image obtained in afocus detection area for detecting the focus state of the object imagepositioned in the frame, a control unit adapted to control the focuscontrol unit to reference at least either an in-focus position obtainedin a first focus detection area in which a person's face is present oran in-focus position obtained in a second focus detection area in whichit is estimated that a person's body is present determining from theposition of the person's face, and a display unit for displaying animage obtained from the image sensors, the focus control unit focusingon and displaying the position of the face detected by the face positiondetection unit if the focus control unit controls the photographingoptical system by referencing the in-focus position in either the firstfocus detection area or the second focus detection area.

According to a fourth aspect of the present invention, a focusadjustment method comprises a face position detection step of detectinga position at which exists at least a face of a person in the frameusing image signals obtained from image sensors which photoelectricallyconvert an object image captured by a photographing optical system, afocus control step of controlling the photographing optical system byreferencing an in-focus position of the object image obtained in a focusdetection area for detecting the focus state of the object imagepositioned in the frame, a control step of controlling such that in thefocus control step at least either an in-focus position obtained in afirst focus detection area in which a person's face is present or anin-focus position obtained in a second focus detection area in which itis estimated that a person's body is present determining from theposition of the person's face is referenced, and a determination step ofdetermining the reliability with which the person's face can be focusedon in the first focus detection area, the control step controlling suchthat the focus control step references either the in-focus positionobtained in the first focus detection area or the in-focus positionobtained in the second focus detection area based the determinationresults of the determination step.

According to a fifth aspect of the present invention, a focus adjustmentmethod comprises a face position detection step of detecting a positionat which exists at least a face of a person in the frame using imagesignals obtained from image sensors which photoelectrically convert anobject image captured by a photographing optical system, a focus controlstep of controlling the photographing optical system by referencing anin-focus position of the object image obtained in a focus detection areafor detecting the focus state of the object image positioned in theframe, and a control step of controlling such that in the focus controlstep at least either an in-focus position obtained in a first focusdetection area in which a person's face is present or an in-focusposition obtained in a second focus detection area in which it isestimated that a person's body is present determining from the positionof the person's face is referenced, the control step controlling thefocus control step to reference the in-focus position obtained in thefirst focus detection area if the difference between the in-focusposition in the first focus detection area and the in-focus position inthe second focus detection area is equal to or less than apre-determined value.

According to a sixth aspect of the present invention, a focus adjustmentmethod comprises a face position detection step of detecting a positionat which exists at least a face of a person in the frame using imagesignals obtained from image sensors which photoelectrically convert anobject image captured by a photographing optical system, a focus controlstep of controlling the photographing optical system by referencing anin-focus position of the object image obtained in a focus detection areafor detecting the focus state of the object image positioned in theframe, a control step of controlling such that in the focus control stepat least either an in-focus position obtained in a first focus detectionarea in which a person's face is present or an in-focus positionobtained in a second focus detection area in which it is estimated thata person's body is present determining from the position of the person'sface is referenced, and a display step of displaying an image obtainedfrom the image sensors, the position of the face detected in the faceposition detection step being focused on and displayed in the focuscontrol step if the focus control step controls the photographingoptical system by referencing the in-focus position in either the firstfocus detection area or the second focus detection area.

According to a seventh aspect of the present invention, a focusadjustment apparatus comprises a face detection unit adapted to detect aposition at which exists at least a face of an object using imagesignals obtained from image sensors which photoelectrically convert anobject image captured by a photographing optical system, a focusdetection unit adapted to detect a plurality of in-focus positions ofthe photographing optical system in a focus detection area in which itis estimated that the body is present corresponding to the face of theobject detected by the face detection unit, and a control unit adaptedto control movement of the photographing optical system based on atleast one of the in-focus positions obtained in the focus detectionarea, the control unit controlling so as to move the photographingoptical system to the in-focus position corresponding to the farthestdistance, from among the plurality of in-focus positions detected by thefocus detection unit.

According to an eighth aspect of the present invention, a focusadjustment apparatus comprises a face detection unit adapted to detectface information of an object using image signals obtained byphotoelectrically converting an object image captured by a photographingoptical system, an estimation unit adapted to estimate the objectdistance from the face information detected by the face detection unit,a focus detection unit for detecting an in-focus position of thephotographing optical system based on the image signals, and a controlunit adapted to control so as to move the photographing optical systemto the detected in-focus position, the control unit selecting thein-focus position to which to move the photographing optical system fromamong a plurality of in-focus positions based on the estimated objectdistance if a plurality of in-focus positions are detected.

According to a ninth aspect of the present invention, a control methodof a focus adjustment apparatus comprises a face detection step ofdetecting a position at which exists at least a face of an object usingimage signals obtained by photoelectrically converting an object imagecaptured by a photographing optical system, a focus detection step ofdetecting a plurality of in-focus positions of the photographing opticalsystem in a focus detection area in which it is estimated that the bodyis present corresponding to the face of the object detected in the facedetection step, and a control step of controlling movement of thephotographing optical system based on at least one of the in-focuspositions obtained in the focus detection area, the control stepcontrolling so as to move the photographing optical system to thein-focus position corresponding to the farthest distance, from among theplurality of in-focus positions detected in the focus detection step.

According to an tenth aspect of the present invention, a control methodof a focus adjustment apparatus comprises a face detection step ofdetecting face information of an object using image signals obtained byphotoelectrically converting an object image captured by a photographingoptical system, an estimation step of estimating the object distancefrom the face information detected in the face detection step, a focusdetection step of detecting an in-focus position of the photographingoptical system based on the image signals, and a control step ofcontrolling so as to move the photographing optical system to thedetected in-focus position, the control unit selecting the in-focusposition to which to move the photographing optical system from among aplurality of in-focus positions based on the estimated object distanceif a plurality of in-focus positions are detected.

According to an 11th aspect of the present invention, a focus adjustmentapparatus comprises a face position detection unit adapted to detect aposition at which exists at least a face of a person in a frame using animage signal obtained from an image sensor which photoelectricallyconverts an object image captured by a photographing optical system, afocus detection unit adapted to detect an in-focus position of thephotographing optical system in at least one of a first focus detectionarea in which the person's face detected by the face position detectionunit is present and a second focus detection area in which it isestimated that the person's body corresponding to the person's facedetected by the face position detection unit is present, and a controlunit adapted to control a movement of the photographing optical systembased on at least one of in-focus positions obtained from the first andsecond focus detection areas, wherein the second focus detection area islarger than the first focus detection area.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a constitution of a digital cameraaccording to a first embodiment of the present invention.

FIGS. 2A and 2B are flow charts showing overall operation of a digitalcamera of the first embodiment.

FIG. 3 is a flow chart showing details of the photometry and focusdetection routine in FIGS. 2A and 2B.

FIG. 4 is a view showing an example of a focus detection frame.

FIG. 5 is a view showing one example of a relationship between a focuslens position and an AF evaluation value signal.

FIG. 6 is a view for explaining the concept of reliability determinationof an AF evaluation value signal.

FIG. 7 is a flow chart showing an AF procedure in the first embodiment.

FIG. 8 is a flow chart for describing operations related to confirmingmonotonic decrease towards the maximum focal length.

FIG. 9 is a flow chart for describing operations related to confirmingmonotonic decrease towards the minimum focal length.

FIG. 10 is a block diagram showing a configuration of an image capturingapparatus of a second embodiment.

FIG. 11 is a flow chart representing operation of the image capturingapparatus of the second embodiment.

FIG. 12 is a flow chart of a sub routine for AF operation in FIG. 11.

FIGS. 13A to 13C are views showing examples of a face detection positionand an AF position.

FIG. 14 is a flow chart of a sub routine for photography operation inFIG. 11.

FIG. 15 is a view showing an evaluation value when a plurality of focuslens in-focus positions are detected in a single focus detection area.

DESCRIPTION OF THE EMBODIMENTS

Below follows a detailed description of preferred embodiments of thepresent invention with reference to the attached drawings.

First Embodiment

FIG. 1 is a block diagram showing a configuration of an image capturingapparatus 100 for performing auto focus using a face detection functionaccording to a first embodiment of the present invention. In the presentembodiment, a digital camera is used as an example for description.

In FIG. 1, reference numeral 101 is an object lens group for capturingan object image on an image sensor 103 described below, and referencenumeral 102 is a light amount adjustment apparatus provided with anaperture stop apparatus and a shutter apparatus. Reference numeral 103is a CCD, CMOS, or other type of image sensor for converting an objectimage captured by the object lens group 101 into an electric signal.Reference numeral 104 is an analog signal processing circuit forperforming such processes as clamping and gaining on the analog signaloutput of the image sensor 103. Reference numeral 105 is ananalog/digital (hereafter “A/D”) converter for converting output of theanalog signal processing circuit 104 into a digital signal. The dataoutput from the A/D converter 105 passes through a digital signalprocessing circuit 107 and a memory control circuit 106, or directlythrough a memory control circuit 106, described below, and is written toa memory 108.

Reference numeral 107 is the digital signal processing circuit whichperforms processes such as pixel interpolation and color conversion onthe data from the A/D converter 105 or the memory control circuit 106.The digital signal processing circuit 107 uses the data from the A/Dconverter 105 to compute a brightness value or a value expressing thefocus state of an object, as well as to adjust white balance.

A system control circuit 112 executes control over an exposure controlcircuit 113 and a focus control circuit 114 based on the computationresults of the digital signal processing circuit 107. Specifically, itperforms a focus control process, an exposure control process, and alight adjustment process using an object image passing through theobject lens group 101. The system control circuit 112 has a displaycontrol circuit 117 for controlling the display of a display apparatus110.

The memory control circuit 106 controls the analog signal processingcircuit 104, the A/D converter 105, the digital signal processingcircuit 107, the memory 108, and a digital/analog (hereafter “D/A”)converter 109. Data undergoing A/D conversion in the A/D converter 105passes through the digital signal processing circuit 107 and the memorycontrol circuit 106, or data undergoing A/D conversion in the A/Dconverter 105 directly passes through the memory control circuit 106,and is written to the memory 108.

Data displayed to the display apparatus 110 is written to the memory108, and the data written to the memory 108 is displayed in the displayapparatus 110 described below via the D/A converter 109. The memory 108stores captured still images or moving images. The memory 108 can alsobe used as a work area for the system control circuit 112.

Reference numeral 110 is the display apparatus made up of a liquidcrystal monitor for displaying images generated from image data obtainedthrough photography, and functions as an electronic finder if objectimages obtained using the image sensor 103 are displayed in sequence.The display apparatus 110 can turn display on and off arbitrarily ascommanded by the system control circuit 112, the power consumption ofthe image capturing apparatus 100 being lower when display is turned offas compared with when display is on. The display apparatus 110 displaysoperating statuses of the image capturing apparatus 100, messages, andthe like using characters, images, and the like as commanded by thesystem control circuit 112.

Reference numeral 111 is an interface for exchanging image data andmanagement information pertaining to the image data with memory cards,hard disks, and other external storage media, as well as with peripheraldevices such as computers and printers. The interface 111 may beconnected directly to various types of communications cards ifconstituted in compliance with standards such as for PCMCIA cards, CF(Compact Flash™) cards, and so on. Such communication cards include LANcards, modem cards, USB cards, IEEE 1394 cards, P1284 cards, SCSI cards,PHS and other communication cards, and so on.

Reference numeral 116 is an orientation detection circuit which detectsthe orientation of the image capturing apparatus 100 and outputs thedetection results to the system control circuit 112.

The system control circuit 112 controls overall operation of the imagecapturing apparatus 100. A memory (not shown) inside the system controlcircuit 112 stores constants, variables, programs, and the like foroperation of the system control circuit 112.

The exposure control circuit 113 drives the aperture stop apparatus andthe shutter apparatus of the light amount control apparatus 102. Thefocus control circuit 114 drives a focusing lens and a zoom lens in theobject lens group 101.

An auto focus operation is controlled by the system control circuit 112and the focus control circuit 114. First, a face position is detected bya face detection circuit 120. The face detection circuit 120 registersfeature data of human faces ahead of time, and recognizes human faces inimage data by comparing with the feature data. In the presentembodiment, the face detection circuit 120 detects human faces bydetecting feature portions of a face, such as eyes and a mouth which areregistered as feature data, in the data from the A/D converter 105 orthe data written to the memory 108. In this case, the position of theface and size information are obtained.

Next, a focus detection frame is set as described below by a focusdetection frame setting section 121, and the focus lens in the objectlens group 101 is driven by the focus control circuit 114. While movingthe focus lens, an AF evaluation value computing section 122 computes asignal corresponding to the contrast of the image using image signalscaptured in each of a plurality of focus detection areas. Next, anin-focus position determining section 123 determines an in-focusposition and the focus lens is controlled and driven to that position,and an in-focus frame is displayed by an in-focus frame display section124. When a switch SW1 (131), which turns on when a release button ispressed halfway, is operated, the focus operation is performed, and whena switch SW2 (132), which turns on when the release button is fullypressed, is operated, the image is captured and stored.

Computation of the AF evaluation value is performed as follows. Ahorizontal band pass filter is applied to each line in the image datainside the focus detection frame. Next, for each line, the band passoutput signal with the largest absolute value is selected. The selectedsignals are vertically integrated. Through this process, the largesthorizontal contrast is detected, and by performing vertical integration,a signal with an improved S/N ratio is obtained. This signal has thelargest value when in-focus, and an increasingly smaller value as theimage becomes defocused. Accordingly, an in-focus image can be obtainedby detecting the maximum value position for the signal and photographingat that position.

Below, the main operation of the image capturing apparatus of thepresent embodiment is described, with reference to the flow charts inFIGS. 2A and 2B. Note that the program for executing this process isstored in the memory of the system control circuit 112, and is executedunder the control of the system control circuit 112.

FIGS. 2A and 2B are flow charts for describing operation during a mainprocess in the image capturing apparatus 100 according to the presentembodiment.

This process is begun after the power is turned on, for example, afterreplacement of the batteries, and in step S101 the system controlcircuit 112 initializes various flags and control variables, etc., inthe internal memory.

In step S102, the system control circuit 112 makes the default settingof the image display in the display apparatus 110 off.

In step S103, the system control circuit 112 detects whether or not aphotography mode has been selected in which settings are made forphotographing and storing image data by the image capturing apparatus100. If no photography mode is set, the process moves to step S104. Instep S104, the system control circuit 112 executes a process accordingto a selected mode different from the photography mode, and returns tostep S103 after that process is finished.

The system control circuit 112 moves to step S105 if a photography modewas set in step S103, and determines whether or not the remaining poweror the operating condition of the power source presents a problem forthe operation of the image capturing apparatus 100. The system controlcircuit 112 moves to step S107 if it determines that there is a problem,displays a prescribed warning via an image or audio using the displayapparatus 110, and then returns to step S103.

In step S105, the system control circuit 112 moves to step S106 if itdetermines that there is no problem with the power source. In step S106the system control circuit 112 determines whether or not the operatingcondition of the storage medium presents a problem for the operation ofthe image capturing apparatus 100, in particular the recording/playbackoperation of data to/from the storage medium. The system control circuit112 moves to step S107 described above if it determines that there is aproblem, displays a prescribed warning via an image or audio using thedisplay apparatus 110, and then returns to step S103.

The system control circuit 112 moves to step S108 if it determines thatthere is no problem in step S106. In step S108, the system controlcircuit 112 displays a user interface (hereafter “UI”) of the settingsof the image capturing apparatus 100 via an image or audio, using thedisplay apparatus 110. Note that if the image display of the displayapparatus 110 was on, the UI display of the various settings of theimage capturing apparatus 100 via images or audio could be done usingthe display apparatus 110. Settings are thus made by the user.

Next, in step S109, the system control circuit 112 sets the imagedisplay of the display apparatus 110 to on.

Further, in step S110, the system control circuit 112 sets the imagecapturing apparatus 100 to observation display mode so that imagesgenerated based on image data obtained by the image sensor 103 aresequentially displayed. In observation display mode, the datasequentially written to the memory 108 is sequentially displayed to thedisplay apparatus 110, and the display apparatus 110 thereby functionsas an electronic finder.

In step S114, the system control circuit 112 checks if the shutterswitch SW1 is pressed or not, and returns to step S103 if it is not, andmoves to step S115 if the shutter switch SW1 is pressed.

In step S115, the system control circuit 112 causes the exposure controlcircuit 113 to perform exposure control such that the brightness valueof the frame is correct. If needed, a flash apparatus (not shown) can becaused to flash during photography, according to the results of theexposure control at this time. Further, the focus control circuit 114 iscaused to perform focus control such that the area in which a human faceis present is in focus.

In step S116, the system control circuit 112 maintains observationdisplay mode until the shutter switch SW2 is operated.

Next, in step S117, if it is detected that the shutter switch SW1 isreleased without the shutter switch SW2 being pressed, the processreturns to step S103.

On the other hand, if the shutter switch SW2 is pressed in step S117,the process moves to step S119.

In step S119, the system control circuit 112 performs a photographyprocess for writing captured image data to the memory 108. The exposurecontrol circuit 113 drives the aperture stop apparatus according to theexposure control results in step S115, and exposes the image sensor 103by opening the shutter. The flash apparatus is caused to flash asneeded, and the shutter is closed once the set exposure time haselapsed. The charge signal output from the image sensor 103 is writtento the memory 108 as image data for saving, via the A/D converter 105,the digital signal processing circuit 107, and the memory controlcircuit 106. The system control circuit 112 further uses the memorycontrol circuit 106 and the digital signal processing circuit 107 toread the image data for saving written to the memory 108 and to executea vertical add process. The digital signal processing circuit 107 inturn is caused to perform a color process, and image data for display isgenerated and once again written to the memory 108.

In step S120, the display apparatus 110 displays the photographed imageusing the image data for display obtained in step S119.

In step S121, the system control circuit 112 reads the image data forsaving written to the memory 108 and causes the memory control circuit106 and, if needed, the digital signal processing circuit 107 to performvarious image processes. The system control circuit 112 then performs animage compression process, and executes a storage process for writingthe compressed image data for saving to a storage medium.

Once the storage process in step S121 is finished, in step S122 thesystem control circuit 112 checks whether or not the shutter switch SW2is pressed. If the shutter switch SW2 is in a pressed state, the processmoves to step S123, and the system control circuit 112 determines thestate of the continuous shooting flag stored in the internal memory ofthe system control circuit 112 or the memory 108. If the continuousshooting flag is on, the system control circuit 112 moves to step S119in order to perform image capturing continuously, and captures the nextimage. In step S123, if the continuous shooting flag is not on, thesystem control circuit 112 moves to step S122, and repeats the processesof steps S122 and S123 until the shutter switch SW2 is turned off.

After the storage process of step S121, if the shutter switch SW2 isoff, the process moves from step S122 to step S124. In step S124, theprocess waits until the set display time for the photographed imageelapsed before moving to step S125.

In step S125, the system control circuit 112 sets the display apparatus110 to observation display mode and moves to step S126. Thus is itpossible to set observation display mode for the next photographingafter confirming the image photographed in the display apparatus 110.

In step S126, the system control circuit 112 checks whether or not theshutter switch SW1 is in an on state, and if it is, moves to step S117and prepares for the next image capture. In step S126, if the shutterswitch SW1 is off, the process moves to step S103 after finishing theseries of image capturing operations.

Next, FIG. 3 is a flow chart showing details of the photometry and focusdetection routine of step S115 in FIG. 2B.

First, when the shutter switch SW1 is turned on in step S114 of FIG. 2A,the system control circuit 112 causes the exposure control circuit 113in step S301 to perform exposure control such that the brightness valuein the frame is correct.

Next, in step S303, the face detection circuit 120 performs detection ofhuman faces. Many methods are publicly known as technologies fordetecting faces. For example, there are methods which use learning, ofwhich neural networks is one typical example. There are also methods forrecognizing parts having features of physical shapes such as the eyes,the nose, the mouth, and the profile of the face using template matchingin the image data. Besides these, there are methods which detect theamount of features in image data such as skin color and the shape of theeyes and use statistical analysis. (See, for example, Japanese PatentApplication Laid-Open No. H10-232934 or Japanese Patent ApplicationLaid-Open No. 2000-48184, etc.) Further, there are methods fordetermining whether or not the immediately preceding face area is nearthe detected area, determining the colors near the face area in order toadd the color of the clothes, or setting the threshold for recognizingfaces lower for the position closer to the center of the frame. Thereare also methods for designating ahead of time the area where the mainobject is present, storing the histogram and color information, andtracking the main object by finding the correlation value. In thepresent embodiment, a face recognition process is performed using amethod for detecting a pair of eyes, a nose, a mouth, and the profile ofa face, and determining a human face based on the relative positionsthereof. Moreover, it is also possible to detect the human faceperiodically from before the SW1 is turned on, and read the most recentface detection results once the SW1 is turned on.

Next, in step S305, the focus detection frame setting section 121 sets afocus detection frame at the position of the body estimated from theposition of the detected face and the face size. For example, in FIG. 4,the position and size of the face obtained from the face detectioncircuit can be applied as the AF frame as-is for the position of theface. As regards the position of the body, assuming there is a bodybelow the face, the frame can be set at the same size as the size of theface frame, immediately below the face frame. Moreover, while the bodyis normally positioned below the position of the face, determining whichway is down with regard to the position of the face is done by detectingthe orientation of the image capturing apparatus using the orientationdetection circuit 116 shown in FIG. 1. Alternately, the tilt of the facein the frame can be detected, thereafter detecting in which directionthe body is positioned. The focus detection frame for the position ofthe body is set so as not to overlap with the focus detection frame ofthe face. One reason for failure to detect focus at the position of theface is that a large portion of the area of the position of the face istaken up by skin color, which has a low difference in brightness values,which means that an AF evaluation value cannot be obtained which hashigh reliability, as discussed below. If the focus detection frame isset at the position of the torso (body), avoiding the position of theface, the possibility rises of obtaining an AF evaluation value withhigh reliability, although it depends on the pattern of the clothes.Also, if not one but a plurality of focus detection frames are set forthe position of the body, it is possible to raise the possibility ofobtaining a focus detection frame which will successfully detect thefocus.

In step S307, the AF evaluation value computing section 122 detects thefocus of the focus detection frames.

Next, in step S309, the in-focus position determining section 123determines whether or not focus was successfully detected in the faceposition frame, and if so the focus control circuit 114 drives the focuslens in step S311 by referencing the in-focus position at the faceposition. If focus is not detected successfully, the focus controlcircuit 114 drives the focus lens in step S313 by referencing thein-focus position of the body position frame. If focus is notsuccessfully detected even at the position of the body, the focus lensis driven to the default position. If focus is successfully detected atboth the frame for the face position and the frame for the bodyposition, the focus lens may be driven, taking into account bothin-focus positions. Note that the determination of whether focus can bedetected or not can be done, for example, by determining whether or notthe difference between minimum and maximum values of the AF evaluationvalue during focus detection are equal to or greater than a prescribedvalue. This is discussed in greater detail below.

Moreover, if a plurality of faces is detected in the frame, focusdetection is performed for each face and body.

After driving the focus lens, in step S315 the in-focus frame displaysection 124 displays the in-focus display frame on the face detectionposition as shown in FIG. 4. In the case of a camera equipped with aface detection function, if a face is successfully detected and it isdetermined that a person can be focused on, an in-focus display on theface position makes it easier for the user to understand that the personis focused on than an in-focus display on the position actually measuredby distance (to the body, etc.). It is also possible to display adisplay different from the focus detection frame for the face positionto a part of the body of the person, in order to make it clear to theuser that the body of the person is focused on.

Next, in step S317, focused on the object, the system control circuit112 once again causes the exposure control circuit 113 to performexposure control such that the brightness value of the frame is correct.Then the system control circuit 112 returns to the main routine in FIGS.2A and 2B.

Next, a method is described for determining whether or not focus wassuccessfully detected in step S309 of FIG. 3.

In step S307 of FIG. 3, the system control circuit 112 performs ascanning AF (auto focus) process in order to detect the in-focusposition. A summary of this is given using FIG. 5. Scanning AF isperformed by obtaining the position of the focus lens at which thehigh-frequency component output by the image signal generated by theimage sensor 103 is highest. The system control circuit 112 moves thefocus lens from the position equivalent to a maximum focal length (“A”in FIG. 5) to the position equivalent to a minimum focal length (“B” inFIG. 5) set in each photography mode. While driving the focus lens, theoutput from the AF evaluation value computing section 122 (the AFevaluation value signal) is obtained, the position at which it ishighest (“C” in FIG. 5) is obtained from the AF evaluation value signalobtained once the driving of the focus lens is finished, and the focuslens is driven to that position.

Obtaining the output of the AF evaluation value computing section 122 isnot performed at all stopped positions of the focus lens in order tospeed up the scanning AF process, and is only performed at prescribedsteps. In this case, the AF evaluation value signal is obtained atpoints a1, a2, and a3 shown in FIG. 5. In this case, the in-focusposition C is obtained by calculation based on the point at which the AFevaluation value signal is highest and points before and after it.Moreover, the focus lens positions at which the AF evaluation valuesignal is obtained at each prescribed step shall henceforth be known as“scan points,” the scan point at maximum focal length being 0, and thescan point at the minimum focal length being N in the presentembodiment.

In the present embodiment, before performing interpolation and obtainingthe point at which the AF evaluation value signal is highest (C in FIG.5), the reliability of the AF evaluation value signal is evaluated. Ifthe reliability is sufficient, then the point at which the AF evaluationvalue signal is highest is obtained, and in-focus detection isdetermined to have succeeded.

If, as a result of evaluating the reliability of the AF evaluation valuesignal, the reliability is low, the process to obtain the point at whichthe AF evaluation value signal is highest is not performed, and in-focusdetection is determined not to have been successful.

Next, a method for determining the reliability of the AF evaluationvalue signal in the scanning AF process is described in detail.

Aside from special cases of maximum and minimum focal lengthsconflicting, the AF evaluation value signal has a hill shape as shown inFIG. 6, with distance along the horizontal axis and the AF evaluationvalue along the vertical axis. Accordingly, in the present embodiment,the reliability of the AF evaluation value signal is determined bydetermining whether or not the AF evaluation value signal has a hillshape, taking account of the difference between the minimum and maximumvalues of the AF evaluation value signal, the length of portions slopingat a slope equal to or greater than a certain fixed value, and the slopeof the sloping portions.

As shown in FIG. 6, in the present embodiment, point P2 and point P3found to slope from the top of the hill (point P1) are obtained, thewidth between point P2 and point P3 is the width of the hill L, and thesum of the difference SL1 between the AF evaluation values at point P1and point P2 and the difference SL2 between the AF evaluation values atpoint P1 and P3 (i.e., SL1+SL2) is the height SL of the hill.

Below, a description is given of specific operations for determining thereliability of the AF evaluation value signal according to the flowchart in FIGS. 7-9, with reference to FIG. 6.

First, in step S701, the maximum value max and the minimum value min ofthe AF evaluation value output from the AF evaluation value computingsection 122, and the scan point io giving the maximum value areobtained. In step S702, the variable L expressing the width of the hillof the AF evaluation value and the variable SL expressing the height ofthe hill are both initialized to 0.

Next, the scan point io giving the maximum value is checked to seewhether or not it is at a position equivalent to the maximum focallength, or in other words, to see whether or not io=0, and if it is at aposition equivalent to the maximum focal length (“yes” in step S703),step S704 is skipped, and the process moves to step S705. On the otherhand, if it is not at a position equivalent to the maximum focal length(“no” in step S703), the process moves to step S704, and the monotonicdecrease towards the focus lens position equivalent to maximum focallength is checked.

Now the process for checking the monotonic decrease towards the focuslens position equivalent to maximum focal length in step S704 isdescribed, with reference to the flow chart in FIG. 8.

First, in step S801, a counter variable i is initialized to io. Then avalue d[i] of the AF evaluation value is compared with a value d[i−1] ofthe AF evaluation value at scan point i−1 closer to the maximum focallength by one scan point from i (a prescribed step). If d[i] is largerthan d[i−1], (“yes” in step S802), monotonic decrease towards themaximum focal length is determined as occurring, the process moves tostep S803, and the variable L expressing the width of the hill of the AFevaluation value and the variable SL expressing the height of the hillare updated according to the following formulas.

L=L+1

SL=SL+(d[i]−d[i−1])

Then in step S802, if d[i]>d[i−1] does not hold true, it is determinedthat monotonic decrease towards the maximum focal length is notoccurring, the process for checking monotonic decrease towards themaximum focal length is finished, and the process moves to step S705.

After the process in step S803, the process moves to step S804, in whichi=i−1, and the point for detection is moved towards the maximum focallength by one scan point.

In steps S805 and S806, L and SL are compared with threshold values Loand SLo related to the width and height of the hill for deeming that itis a hill, and determined whether or not they are equal to or greaterthan the threshold values. If they are both equal to or greater than thethreshold values, the conditions in the process for determining thereliability of the AF evaluation value performed in steps S708 and S709in FIG. 7, discussed below, are already met, so the process for checkingmonotonic decrease towards the maximum focal length is not performed,and the process moves to step S705.

In the case of “no” in step S805 or S806, the process moves to stepS807, and the counter i is checked to see whether or not it has reacheda value equivalent to the maximum focal length (=0). If the value of thecounter i is not 0, then the process returns to step S802 and the aboveprocesses are repeated. If it is 0, or in other words, if the scan pointhas reached the position of the maximum focal length, the process forchecking for monotonic decrease towards the maximum focal length isfinished, and the process moves to step S705.

Thus, monotonic decrease towards the maximum focal length is checkedfrom i=io.

If the process for checking for monotonic decrease towards the maximumfocal length finished in step S704, next the scan point io giving themaximum value max is checked to see whether it is at a positionequivalent to the minimum focal length (N) for performing scanning AF,and if it is at a position equivalent to the minimum focal length (“yes”in step S705), then step S706 is skipped, and the process moves to stepS707. On the other hand, if it is not at a position equivalent to theminimum focal length (“no” in step S705), the process moves to stepS706, and the monotonic decrease towards the focus lens positionequivalent to minimum focal length is checked.

Now the process for checking the monotonic decrease towards the focuslens position equivalent to minimum focal length in step S706 isdescribed, with reference to the flow chart in FIG. 9.

First, in step S901, the counter variable i is initialized to io. Thenthe value d[i] of the AF evaluation value at scan point i is comparedwith the value d[i+1] of the AF evaluation value at scan point i+1closer to the minimum focal length by one scan point from i. If d[i] islarger than d[i+1], (“yes” in step S902), monotonic decrease towards theminimum focal length is determined as occurring, the process moves tostep S903, and the variable L expressing the width of the hill of the AFevaluation value and the variable SL expressing the height of the hillare updated according to the following formulas.

L=L+1

SL=SL+(d[i]−d[i+1])

Then in step S902, if d[i]>d[i+1] does not hold true, it is determinedthat monotonic decrease towards the minimum focal length is notoccurring, the process for checking monotonic decrease towards theminimum focal length is finished, and the process moves to step S707.

After the process in step S903, the process moves to step S904, in whichi=i+1, and the point for detection is moved towards the minimum focallength by one scan point.

In steps S905 and S906, L and SL are compared with threshold values Loand SLo related to the width and height of the hill for deeming that itis a hill, and determined whether or not they are equal to or greaterthan the threshold values. If they are both equal to or greater than thethreshold values, the conditions in the process for determining thereliability of the AF evaluation value performed in steps S708 and S709in FIG. 7, discussed below, are already met, so the process for checkingmonotonic decrease towards the minimum focal length is not performed,and the process moves to step S707.

In the case of “no” in step S905 or S906, the process moves to stepS907, and the counter i is checked to see whether or not it has reachedthe value equivalent to minimum focal length (=N). If the value of thecounter i is not N, then the process returns to step S902 and the aboveprocesses are repeated. If it is N, or in other words, if the scan pointhas reached the position of the minimum focal length, the process forchecking for monotonic decrease towards the minimum focal length isfinished, and the process moves to step S707.

Thus, monotonic decrease towards the minimum focal length is checkedfrom i=io.

If the checks for monotonic decrease towards the maximum and minimumfocal lengths are finished, all coefficients for determining thereliability of the AF evaluation value are compared to their respectivethreshold values, and if all the conditions are met, the AF evaluationvalue is determined to be reliable.

In step S707, the difference between maximum value max and minimum valuemin of the AF evaluation value is compared to a threshold value, and ifit is smaller than the threshold value, reliability is determined to belacking, and the process moves to step S711. Note that this thresholdvalue is set low so that detection is only successful in cases where thecontrast is definitely low. If the answer in step S707 is “yes,” in stepS708 the length L of the portion sloping at a slope equal to or greaterthan a certain fixed value is compared with a threshold value Lo, and ifit is smaller than the threshold value Lo, reliability is determined tobe lacking, and the process moves to step S711. If the answer in stepS708 is “yes,” then in step S709, the height SL is compared with SLo,and if this is smaller than the prescribed value, reliability isdetermined to be lacking, and the process moves to step S711.

If all three of the above conditions are met, the AF evaluation value isdetermined to be reliable, and the process moves to step S710, and theposition is obtained for driving the focus lens from the AF evaluationvalue computed by the AF evaluation value computing section 122. This iscalculated by obtaining, through interpolation, etc., the position atwhich the discretely calculated AF evaluation value is highest.

If the AF evaluation value is determined not to be reliable and theprocess moves to step S711, it is determined that focus was notsuccessfully detected.

As described above, according to the above embodiment, even in caseswhere the contrast on the face is low and focus cannot be detected, oreven in cases where objects with differing distances such as faces andthe background are mixed in the focus detection area, it is possible tofocus on a person with accuracy.

Note that in the above embodiment, the focus lens is driven to thein-focus position of the face position if focus is detected successfullyat the face position. However, there exists the possibility of the focusdetection result at the face position resulting in the focus being toofar back due to being affected by the contrast of the background. Forthis reason, it is possible to determine the focus detection results atthe face position correct and to drive the focus lens to the in-focusposition at the face position only if the difference between thein-focus positions at the face position and at the body position areequal to or less than a prescribed value, and to drive the focus lens tothe in-focus position of the body position if the difference is largerthan the prescribed value.

Note that if the face is not successfully detected, it is also possibleto perform focus detection by setting the focus detection frame in thecenter of the frame, for example.

Second Embodiment

A second embodiment is described below.

(Abbreviated Constitution of a Digital Camera)

FIG. 10 is a view showing an abbreviated constitution of a digitalcamera for performing focus operations using the face detectionfunction.

The digital camera has a photographing optical system made up of anoptical system 1101, a focus lens 1102, and so on. Light captured by thephotographing optical system (the object image) is photoelectricallyconverted by an image sensor 1103. A signal digitized via an A/Dconverter 1105 and a pre-processing circuit 1104 provided with anon-linear amplifying circuit and a CDS circuit for eliminating outputnoise is stored in a memory 1107 via a memory controller 1106. This isconverted into an image by a signal processing circuit 1108 and storedon a storage medium 1109. Note that 1110 is an operation display portionfor displaying a captured frame and an AF frame (a frame showing thearea corresponding to the focus detection area on the image sensor) whencapturing an image, as well as for displaying images and camera stateinformation. General display control of these displays is done by asystem control section 1113. 1111 is a switch (hereafter “SW1”) forgiving a photography standby operation command, and 1112 is aphotography switch (hereafter “SW2”) for giving a photography commandafter the SW1 is operated.

Focusing operation is controlled by the system control section 1113.First, a face detection section 1115 detects face information includinga face position. Regarding the method of detecting the face, a publiclyknown method can be used. In this case, the position of the face andsize information are obtained. Next, the AF frame (the focus detectionarea in the image sensor) is set by an AF frame setting section 1116 asdescribed below, and a focus lens 1102 is moved by a focus lens drivecircuit 1114. While moving the focus lens, captured image signalscaptured at a plurality of focus lens positions are used to compute asignal corresponding to the contrast of the captured image signal (asignal expressing the focus state, etc.) in an AF evaluation valuecomputing section 1117. Next, an in-focus position determining section1118 determines an in-focus position and the focus lens is controlledand driven to that position, and an in-focus frame is displayed to theoperation display section 1110 by an in-focus frame display section1119. Note that AF and other photography standby operations areperformed by operating the SW1, and that operation of the SW2 causesphotography and storage of an image to be performed.

Computation of the AF evaluation value is performed as follows. Ahorizontal band pass filter is applied to each line in the image datainside the focus detection area. Next, for each line, the band passoutput signal with the largest absolute value is selected. The selectedsignals are vertically integrated. With the above constitution, thelargest contrast in the horizontal direction is detected, andintegrating this vertically obtains a signal with an improved S/N ratio.This signal has the largest value in an in-focus state, and the valuedecreases in unfocused states. Accordingly, an in-focus image can beobtained by detecting the focus lens position where the signal is at itsmaximum value and photographing at that position.

(Process of Operation of the Digital Camera)

A detailed description of the operation of the digital camera is givenbelow with reference to the flow chart in FIG. 11.

First, the state of the main switch for supplying power to the system(not shown) is detected in step S1201, and if this is on, the processmoves to step S1202. In step S1202, the remaining capacity of thestorage medium 1109 is checked, and if this is 0 (zero), the processmoves to step S1203, and if not, to step S1204. In step S1203, a warningthat the remaining capacity of the storage medium 1109 is 0 is provided,and the process returns to step S1201. The warning may be displayed tothe operation display section 1110, via a warning sound from an audiooutput section (not shown), or by both.

In step S1204, the state of the switch SW1 is checked, and if it is on,the process moves to step S1206, and if not, to step S1205. In stepS1205, the state of a main switch (not shown) is checked, and if it ison, the process moves to step S1204, and if not, to step S1201. In stepS1206, an AE process is performed, and in step S1207, an AF operation isperformed according to the flow chart in FIG. 12 discussed below. Instep S1208, the state of the SW2 is checked, and if it is on, theprocess moves to step S1210, and if not, to step S1209. In step S1209,the state of the SW1 is checked, and if it is on, the process returns tostep S1208, and if not, to step S1204. In step S1210, a photographyoperation is performed according to the flow chart in FIG. 14 discussedbelow. In step S1211, the remaining capacity of the storage medium 1109is checked, and if this is 0 (zero), the process moves to step S1203,and if not, to step S1212. In step S1212, the state of the SW2 ischecked, and if it is not on, then the process moves to step S1209.

(AF Operation if Only One Face is Detected)

The AF operation sub-routine in step S1207 in the flow chart in FIG. 11is described below, with reference to the flow chart in FIG. 12.

In step S1301, face detection is performed by a face detection section1115. In step S1302, the size of the AF frame is determined by thedetected face size, and a plurality of AF frames are set within apossible range in the photography frame at the detected face positionand along a plumb line direction along which a body is expected to bepresent with regard to the face position.

For example, as shown in FIG. 13A, an AF frame is set at the positionand size of the face obtained by the face detection section 1115 withregard to the face position. Further, a plurality of AF frames of thesame size as the AF frame set for the face position are lined up and setif possible down to the position of the chest and abdomen of the faceobject along a plumb line direction from the face position along which abody is expected to be positioned.

At this time, depending on the position and size of the detected face,it might not be possible to set AF frames along a direction along whicha body is expected to be present with respect to the face positioninside the photography frame, and therefore an AF frame is only set atthe face position.

In step S1303, a focus detection operation is performed in the focusdetection area corresponding to the AF frame. In step S1304, adetermination is made as to whether or not an in-focus position wassuccessfully obtained by detecting focus in the focus detection areacorresponding to the AF frame at the face position. If an in-focusposition is successfully obtained, in step S1305 the focus lens is movedto the in-focus position in the focus detection area corresponding tothe face position AF frame, and if an in-focus position is notsuccessfully obtained, the process moves to step S1306. In step S1306, acheck is made to see if AF frames were set in step S1302 in a directionalong which a body is expected to be present with respect to the faceposition. If AF frames are set in a direction along which the body isexpected to be present with respect to the face, the process moves tostep S1309, and if the AF frame is only set to the face position, thenthe process moves to step S1307. In step S1309, a check is made to seewhether a plurality of AF frames were set in a direction along which abody is expected to be present with respect to the face position forwhich an in-focus position was successfully obtained, and if there is aplurality, then the process moves to step S1310 in which focus detectionis performed and the focus lens is moved from the corresponding focusdetection area to an in-focus position farther away from the object. Ifthere is not a plurality, then the process moves to step S1311. In stepS1311, a check is made to see if an in-focus position was successfullyobtained in the focus detection area corresponding to any one of the AFframes in the direction along which a body is expected to be presentwith respect to the face position, and if it was, then the focus lens ismoved to the in-focus position obtained in step S1312. If no in-focusposition was successfully obtained in the focus detection areacorresponding to any of the AF frames, the process moves to step S1307,the focus lens is moved to a pre-set position called a fixed point, andan un-focused display is performed in step S1308.

In step S1313, in-focus display is performed at the face detectionposition. In the case of a camera equipped with a face detectionfunction, if it is determined that face detection is possible and thatit is possible to detect an in-focus position at the face position ofthe object or immediately below the face position, then in-focus displayis performed at the face position. This is because in-focus display atthe face position makes it easier to understand that the person (theintended object) is focused on than in-focus display on the position forwhich focus is actually detected (the area of the body, etc.).

Note that the determination of whether an in-focus position can beobtained or not can be done, for example, by determining whether or notthe difference between minimum and maximum values of the AF evaluationvalue during focus detection are equal to or greater than a prescribedvalue.

By using the constitution described above, the face object can focusedon even if obstacles are present at closer distances at the bodyposition of the detected face object as shown in FIGS. 13A and AF is notpossible at the face position.

(Photography Operation)

The photography operation sub-routine in step S1210 in the flow chart inFIG. 11 is described below, with reference to the flow chart in FIG. 14.

First, the brightness of the object is measured in step S1501. In stepS1502, exposure is performed on the image sensor 1103 in accordance withthe object brightness measured in step S1501. The image captured on thesurface of the image sensor 1103 is photoelectrically converted andbecomes an analog signal, is sent to the A/D converter 1105 in stepS1503, and is converted to a digital signal after pre-processing, suchas elimination of output noise from the image sensor 1103, non-linearprocessing, etc. The output signal from the A/D converter 1105 is madeinto a correct output image signal in step S1504 by a signal processingcircuit 1108, the output image signal is converted into an image formatsuch as JPEG format in step S1505, and is transferred and stored in astorage medium 1109 in step S1506.

A constitution is thus achieved in which an in-focus position can beobtained by setting an AF frame (focus detection area) at a positionwhich does not include the detected face even in cases where the focuscannot be adjusted because the face contrast is too low. By detectingfocus at the position of the body measured from the detected face inthis fashion, it is possible to adjust focus to the detected face objecteven if the face contrast is too low. It is possible to focus on theintended object without adjusting the focus on obstacles even if closerobstacles are present in the body position of the detected face object.

Note that in the above description, the intended object was described asa person, but this is not a limitation. For example, cases are alsoconceivable in which the object image is clipped from the background. Inthis case, the object may be an object other than a person whose size isto a certain extent known, and whose position of area other than theclipped detected area is known.

In the above description, a constitution was used in which the focusdetection results are not referenced in the focus detection areascorresponding to a plurality of AF frames in a direction along which abody is expected to be present with respect to the face position whenfocus is successfully detected at the face position. However, it is alsopossible to estimate an in-focus position for the focus lens using boththe focus detection results at the face position and the focus detectionresults at a position along which the body is expected to be presentwith respect to the face position.

(AF Operation if a Plurality of Faces are Detected)

Next, AF operation when a plurality of faces are detected is described,with reference to the flow chart in FIG. 12. The method for setting AFframes when there is a plurality of face positions detected by the facedetection section 1115 is different from the above. Other aspects arethe same, so description of their operation is omitted.

In step S1301, face detection is performed by a face detection section1115. In a case in which there are two or more faces detected, a mainface selection process is implemented for selecting a face which is tobe the main object (a main face). Selection of the face to be the mainface is done taking into consideration information regarding past facesacting as main faces, reliability indicating the accuracy of thedetected face, the size of the face, the position of the face, and soon. Two other pieces of information are obtained: information on theface to be the main object and information on other detected faces.

When a plurality of faces are detected, AF frames are set and focusstates are detected for as many of the faces as possible, thus making iteasier to check whether or not the focus is on the user's intendedobject. Accordingly, in step S1302, first the size of an AF frame isdetermined based on the main face size. Next, the vertical andhorizontal intervals of the AF frames are set, giving priority topositioning the AF frames for the other plurality of detected faces.Next, AF frames are set in a direction along which bodies are expectedto be present with respect to each detected face position. In stepS1303, focus detection operations are performed in the focus detectionareas corresponding to the AF frames. In step S1304, a determination ismade as to whether or not an in-focus position was successfully obtainedby detecting focus in the focus detection area corresponding to the AFframe at the main face position. If an in-focus position is successfullyobtained, the focus lens is moved to the in-focus position at the focusdetection area corresponding to the AF frame of the main face positionin step S1305. If an in-focus position was not successfully obtained inthe focus detection area corresponding to the AF frame of the main faceposition, the process moves to step S1306.

Note that in the above, the main face and (the AF result of) thedependent frame thereon is the subject of the focus detection resultreferenced during focus lens control, and that frames for faces otherthan the main face are used only for deciding whether or not to bring upthe in-focus display.

In this case, if the interval of the AF frames is set to 0, the AF framepositioned immediately below the face position is often set to theposition of the neck of the face object, and the results of obtainingthe in-focus position through focus detection are very frequently anin-focus position farther away from the face object due to near-farconflict with the background. For this reason, the focus detection areain the direction in which the body is expected to be present withrespect to the face position associated with that face is made to be theAF frame two below the face position AF frame (skipping one), and thefocus detection area corresponding to the AF frame one further belowthat (FIG. 13B). In a case in which there is an interval between AFframes, the focus detection area in the direction in which a body isexpected to be present with respect to the face position associated withthe face is made to be the focus detection area corresponding to the AFframe immediately below and adjacent to the face position AF frame, andthe focus detection area corresponding to the AF frame immediately belowthat (FIG. 13C).

In step S1306, a check is made to see if AF frames were set in stepS1302 in a direction along which a body is expected to be present withrespect to the main face position. If AF frames are set at positionsalong which a body is expected to be present with respect to the mainface, the process moves to step S1309, and if an AF frame is only set tothe face position and no AF frames are set to positions along which abody is expected to be present with respect to the main face, theprocess moves to step S1307. In step S1309, a check is made to see ifthere is a plurality of AF frames in a direction along which a body isexpected to be present with respect to the main face position for whichan in-focus position was successfully obtained. If there is a plurality,the process moves to step S1310 and focus detection is performed, andthe focus lens is moved to the in-focus position furthest from theobject from among the focus detection areas corresponding to theplurality of AF frames corresponding to the plurality of AF frames forwhich in-focus positions were successfully obtained. If there is not aplurality, then the process moves to step S1311. In step S1311, a checkis made to see if an in-focus position was successfully obtained in afocus detection area corresponding to one of the AF frames in thedirection along which a body is expected to be present with respect tothe main face position. If it was successfully obtained, then the focuslens is moved to the in-focus position obtained in step S1312. If anin-focus position was not successfully obtained even in a focusdetection area corresponding to any of the AF frames in the directionalong which a body is expected to be present with respect to the mainface position, the process moves to step S1307. The focus lens is movedto a pre-set position called a fixed point, and unfocused display isperformed in step S1308.

In step S1313, in-focus display is performed at the main face detectionposition. In the case of a camera equipped with a face detectionfunction, if it is determined that face detection is possible and thatit is possible to detect an in-focus position at the face position ofthe main object or the position immediately below the face positionwhere a body is expected to be present with respect to the faceposition, then in-focus display is performed at the face position. Thisis because in-focus display at the face position makes it easier tounderstand that the person (the intended object) is focused on thanin-focus display on the position for which focus is actually detected(the area of the body, etc.). When performing in-focus display of themain face frame, the face frame is also displayed in-focus if thedifference between the AF result for the AF frame for the main face andthe AF result for the AF frame associated with face frames other thanthe main face is within a predetermined depth.

By performing focus detection and in-focus display through control asdescribed above, it is possible to clearly inform the user which objectis being focused on.

In the above description, the main face is selected first, and thenfocus adjustment is made so as to focus on the main face (or on aposition where a body is expected to be present with respect to the mainface if focus detection on the main face was not possible). Aconstitution has been adopted wherein once an in-focus state isachieved, an in-focus display is performed. However, this is not alimitation, and a constitution may be adopted in which AF results areused which are the furthest away, regardless of whether they are for themain face or another face, from among the results of the AF framesetting and focus detection, as shown in FIG. 13B and FIG. 13C. Aconstitution may also be adopted to determine which AF results to use,according to a priority order of faces which has been pre-detected.

(AF Operation when Using Information Estimating the Object Distance fromthe Face Size)

Next, a description is given of using information wherein the objectdistance is estimated from the face size in the selection method of theAF results in the focus detection areas corresponding to the pluralityof AF frames associated with the main face. Other aspects are the same,so description of their operation is omitted.

Next, in step S1310 of the flow chart in FIG. 12 discussed above, aconstitution is adopted in which the focus lens is moved based on the AFresults of the focus detection area corresponding to the AF frame at adistance within the object field depth with regard to the estimateddistance calculated from the face size of the main face, and closer tothe estimated distance.

The estimated distance is obtained as follows from the face sizedetected by the face detection section 1115. Assuming an actual standardface size to be Lstd, the size of the face area detected on the image tobe L, and the focal length to be f, the estimated distance d can beobtained with d=f(1+Lstd/L)

Further, assuming the diameter of the permissible circle of confusion tobe ∂, the F stop of the optical system to be F, then the rear distancedB and the front distance dC entering the depth with regard to thein-focus distance dA can be obtained with

dB=dA+FδdA2/(f2−dAFδ)

dC=dA+FδdA2/(f2+dAFδ)

Accordingly, assuming the rear distance and the front distance enteringthe depth with respect to the estimated distance d obtained from theface size to be db and dc, respectively, and the AF results in the focusdetection areas corresponding to the plurality of AF frames associatedwith the face to be dj (j=0, 1, . . . , N), determination is made usingthe following formula.

dc<dj<db

By making this determination, of the AF results in the focus detectionareas corresponding to the plurality of AF frames associated with theface, the AF results outside the object field depth of the face can beeliminated, and the AF results dj closer than the estimated distance dcan be selected. In terms of the effect, erratic AF can be avoided inscenes in which the body of the face object is not positioned in aposition in which the body is expected to be present with respect to thedetected main face.

In the above description, it was made possible to select only those AFresults within the object field depth with regard to the estimateddistance obtained on the basis of the face size, but a constitution mayalso be adopted in which the AF results within a range which is a timesthe object field depth (within a predetermined range) are adopted.

Note that in the above description, the focus lens is driven to the AFresult of the face position if AF is successful at the face position.However, there exists the possibility of the result at the face positionresulting in the focus being too far back due to being affected by thecontrast of the background. For this reason, a constitution may beadopted in which the AF result at the face position is only deemedcorrect when the AF distances at the face position and at the positionof the body corresponding thereto are close, and the focus lens is movedto the AF result for the face position. In this case, a constitution isadopted in which if the AF distances for the face position and for thebody position corresponding thereto are not close, the focus lens isdriven according to the AF result of the body position.

Note that if the face is not successfully detected, it is also possibleto perform AF by setting the focus detection area in the center of theframe, for example.

Moreover, in the description above, an image capturing apparatus havingan image capturing optical system such as a zoom lens and a focus lenswas described as an example of a focus adjustment apparatus, but this isnot a limitation. For example, focus adjustment apparatuses such astelescopes and microscopes are also possible.

With the constitution as described above, it is possible to focus on theuser's intended object in a groundbreaking fashion compared with thepast, even in cases in which the face contrast is low and AF isimpossible, or cases in which objects with differing distances, asidefrom the face, are present in the focus detection area.

In the above description, a signal in the focus detection area of theimage sensor was obtained, an evaluation value indicating a focus statewas computed, and an in-focus position of the focus lens was detected,but this is not a limitation. For example, it is also possible toprovide a plurality of focus detection areas whose positions correspondto the image sensor on an AF sensor in phase-difference AF, detect afocus state in a focus detection area on the AF sensor corresponding tothe body of the object, and perform movement control of the focus lensbased on the focus state.

In the above description, a constitution was adopted in which aplurality of focus detection areas is provided to positions at which abody is expected to be present with respect to the face of the object,and an evaluation value was detected indicating focus states. Thismatter can also be applied to cases in which an evaluation value,indicating a focus state obtained in a single focus detection areaprovided to a position in which a body is expected to be present withrespect to a face, indicates a plurality of in-focus positions (see FIG.15). In other words, if a plurality of in-focus positions of the focuslens are detected in a single focus detection area corresponding to theposition of the body of an object (near and far in FIG. 15), the focuslens can be moved to the farther in-focus position of the focus lens.This way it is possible to focus on the user's intended object in agroundbreaking fashion compared to the past, even in cases in whichthere are objects with differing distances in front of the body of theobject, such as flower vases, tables, and so on. This is based on thefact that in general the object's body is present in a plumb linedirection from the face of the object, and there is not an empty spacethere. Even if this is not the case, accurate detection is possible bycalculating the estimated object distance from the face size andselecting from within a prescribed depth.

Other Embodiments

The object of these embodiments can be attained by the followingmethods. In other words, a system or apparatus is provided with astorage medium (or a recording medium) in which is stored program codeof software for realizing the functionality of the embodiments describedabove. A computer (or CPU or MPU) of that system or apparatus reads andexecutes the program code stored in the storage medium. In this case,the program code read from the storage medium realizes the functionalityof the embodiments described above, and the storage medium on which isstored the program code constitutes the present invention. Not only isthe functionality of the embodiments described above realized byexecuting the program code read by the computer, but the following casesare also included in the invention. In other words, an operating system(OS) or the like running on the computer performs all or part of theactual processes based on the instructions in the program code, and thefunctionality of the embodiments described above is realized throughthose processes.

Further, the following cases are also included in the invention. Theprogram code read from the storage medium is read into memory providedto a function enhancing card inserted into the computer or a functionenhancing unit connected to the computer. Cases in which a CPU, etc.,provided to the function enhancing card or the function enhancing unitperforms all or part of the actual processes based on the instructionsof the program code and the functionality of the embodiments describedabove is realized by those processes are also included.

When applying the present invention to the above storage medium, theprogram code corresponding to the procedure described above is stored onthat storage medium.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2006-131810, filed May 10, 2006, and No. 2006-249748 filed Sep. 14,2006, which are hereby incorporated by reference herein in theirentirety.

1-19. (canceled)
 20. A focus adjustment apparatus comprising: a facedetection unit that detects a position at which at least a face of aperson exists from an image signal obtained by photoelectricallyconverting an object image formed by a photographing optical system; afocus detection unit that detects a plurality of in-focus positions ofthe photographing optical system in a focus detection area in which itis estimated that an object's body, which corresponds to the face of theobject detected by said face detection unit, is present; and a controlunit that controls movement of the photographing optical system based onat least one of the in-focus positions obtained in the focus detectionarea, wherein said control unit controls to move the photographingoptical system to the in-focus position corresponding to a fartherdistance in the plurality of in-focus positions detected by said focusdetection unit.
 21. The apparatus according to claim 20, wherein saidfocus detection unit sets a plurality of focus detection areas in whichit is estimated that an object's body which corresponds to the face ofthe object detected by said face detection means, and detects thein-focus position of the photographing optical system for each of theplurality of focus detection areas.
 22. The apparatus according to claim20, further comprising an estimating unit that estimates an objectdistance from a size of the face of the object detected by said facedetection unit, wherein said control unit deviates an in-focus positionin the plurality of in-focus positions detected by said focus detectionmeans, which is not an in-focus position within the depth ofphotographing field of the object distance obtained by said estimatingunit, from the target focus position.
 23. The apparatus according toclaim 20, wherein said focus detection unit detects the in-focusposition of the photographing optical system in the focus detection areain which it is estimated that an object's body, which corresponds to theface of the object, is present in a case that the in-focus position ofthe photographing optical system cannot be detected in the focusdetection area in which the face of the object detected by said facedetection unit exists.
 24. The apparatus according to claim 20, furthercomprising a display control unit that controls a display unit todisplay an in-focus display at a corresponding face position in a casethat said control unit controls to move the photographing optical systemto the in-focus position of the photographing optical system in thefocus detection area in which it is estimated that an object's body,which corresponds to the face of the object, is present.
 25. An imagecapturing apparatus comprising: an image sensor that outputs an imagesignal; and the focus adjustment apparatus according to claim
 20. 26. Acontrol method of a focus adjustment apparatus, said method comprising:a face detection step of detecting a position at which at least a faceof a person exists from an image signal obtained by photoelectricallyconverting an object image formed by a photographing optical system; afocus detection step of detecting a plurality of in-focus positions ofthe photographing optical system in a focus detection area in which itis estimated that an object's body, which corresponds to the face of theobject detected in said face detection step, is present; and a controlstep of controlling movement of the photographing optical system basedon at least one of the in-focus positions obtained in the focusdetection area, wherein said control step controls to move thephotographing optical system to the in-focus position corresponding to afarther distance in the plurality of in-focus positions detected in saidfocus detection step.
 27. A program causing a computer to perform thecontrol method defined in claim
 26. 28. A computer readable storagemedium storing the program defined in claim claim 26.